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DETAILED ACTION 
Claim Objections 

1 . Claim 43 objected to because of the following informalities: The claim appears in 
the same format as claims 30 and 37 but there is a typo which has claim 43 depending 
from 41 instead of claim 42. Appropriate correction is required. 

Claim Rejections - 35 USC §112 

2. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

3. Claims 9, 10, 20, 21, 24 - 37 are rejected under 35 U.S.C. 112, second 
paragraph, as being indefinite for failing to particularly point out and distinctly claim the 
subject matter which applicant regards as the invention. In particular the use of the 
phrase "if necessary" in claims 9, 21, 24, 31 is unclear and needs to be more fully 
fleshed out in the claims. 

4. Claims 10, 21, 25 - 30, 32 - 37 are rejected as they depend from previously 
rejected claims. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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6. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 
USPQ 459 (1966), that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

7. Claims 1 - 1 0, 1 3 - 21 , 24 - 28, 31 - 35 are rejected under 35 U.S.C. 1 03(a) as 
being unpatentable over Uyttendaele (6,701,030) in view of Kanatani ("Optimal 
Homography Computation with a Reliability Measure", IAPR Workshop on Machine 
Vision Applications, Nov. 17- 19, 1998). 

8. In regards to claim 1, Uyttendaele discloses in figure 3A a method for combining 
a plurality of frames of image pixel data. In step 300 Uyttendaele acquires multiple 
frames of image pixel data. In the paragraph starting at column 4, line 45 Uyttendaele 
discloses, "a camera 163 capable of capturing a sequence of images 164 can also be 
included as an input device to a personal computer." Any image inputted into a 
computer must be comprised of pixels. Uyttendaele then discloses in the paragraph 
starting at column 5, line 50 that each frame is "captured by a different cameras from a 
different viewpoint." Uyttendaele discloses in the paragraph starting at column 7, line 41 
that "the lateral field of view of each camera overlaps by at least 20 percent." Returning 
to figure 3A Uyttendaele discloses in step 306 that the image frames are combined 
(mosaic). Uyttendaele discloses in step 308 compensating for any localized ghosting. 
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Uyttendaele however, does not disclose the use of a homography transformation matrix 
to perform this "deghosting" or warping of the images. 

9. Kanatani teaches in the Introduction on page 426 that, "A homography is a 
mapping that occurs between two perspective images of a planar surface in the scene. 
The computation of homographies plays an essential role in image registration and 
mosaicing." 

10. Therefore it would have been obvious to one having ordinary skill in the art at the 
time of the invention to use homography transformation matrices (as taught by 
Kanatani) to assist in the "deghosting" and warping performed in the mosaicing of 
Uyttendaele. Homography allows for the mapping between two points which would be 
necessary for the merging of images performed by Uyttendaele, this could result in a 
cleaner mosaicing with less ghosting. 

11. In regards to claim 2, Uyttendaele discloses in the paragraph starting at column 
7, line 41 that "at least as many cameras as needed to provide a full 360 degree 
coverage of the scene including the desired overlaps would be employed." 

12. In regards to claim 3, Uyttendaele discloses in the paragraph starting at column 
7, line 41 that the "camera overlaps by at least 20 percent". This is about 25 percent. 

13. In regards to claim 4, as shown in claim 3 Uyttendaele discloses overlaps of at 
least 20 percent. As each image would overlap with two other images the total overlap 
for any one image would be about 40 percent. This is less than 85 percent of the field 
of view. 
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14. In regards to claim 5, Kanatani discloses the use of homography matrices, which 
are based on landmark points of commonality as seen in the first full paragraph in the 
second column on page 426. 

15. In regards to claim 6, this claim has already been dealt with in claim 1. The entire 
idea of the invention disclosed by Uyttendaele is to fuse the plurality of frames of image 
pixel data into a single image, as shown in claim 1 this can be done using homography 
matrices. 

16. In regards to claim 7, the subject of overlap percentages has been dealt with in 
claims 3 and 4. In regards to the outer perimeter edge, when Uyttendaele creates the 
panoramic image there is an upper and lower perimeter edge (as a panoramic image 
can be considered a ring, which has two edges) either could be the outer perimeter 
edge. 

17. In regards to claim 8, as shown in claim 3 Uyttendaele discloses overlaps of at 
least 20 percent. As each image would overlap with two other images the total overlap 
for any one image would be about 40 percent. 40 percent is about 35 percent. 

18. In regards to claim 9, Uyttendaele in the paragraph starting at column 7, line 41 
disclosed that a "camera rig preferably consists of multiple digital video cameras that 
are disposed in a back to back fashion such that their lenses each point in a radially 
outward direction and view a different portion of the surrounding scene." Uyttendaele 
does not disclose locating imaging devices at one or more additional installation sites. 
However as Uyttendaele is attempting to obtain a panoramic image it is not necessary 
for the method disclosed by Uyttendaele to have additional installation sites. 
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19. In regards to claim 10, Uyttendaele discloses in the paragraph starting at column 
7, line 41 that "at least as many cameras as needed to provide a full 360 degree 
coverage of the scene including the desired overlaps would be employed." Thus 
inherently Uyttendaele discloses determining the limiting range of at least one of a 
plurality of imaging devices, as this information would be needed to determine the 
desired overlaps. 

20. In regards to claim 13, Uyttendaele discloses in figure 3A a method for combining 
a plurality of frames of image pixel data. In step 300 Uyttendaele acquires multiple 
frames of image pixel data. In the paragraph starting at column 4, line 45 Uyttendaele 
discloses, "a camera 163 capable of capturing a sequence of images 164 can also be 
included as an input device to a personal computer." Any image inputted into a 
computer must be comprised of pixels. Uyttendaele then discloses in the paragraph 
starting at column 5, line 50 that each frame is "captured by a different cameras from a 
different viewpoint." Uyttendaele discloses in the paragraph starting at column 7, line 41 
that "the lateral field of view of each camera overlaps by at least 20 percent." Returning 
to figure 3A Uyttendaele discloses in step 306 that the image frames are combined 
(mosaic). This step requires a computing apparatus. Uyttendaele discloses in step 308 
compensating for any localized ghosting. Uyttendaele however, does not disclose the 
use of a homography transformation matrix to perform this "deghosting" or warping of 
the images. 

21. Kanatani teaches in the Introduction on page 426 that, "A homography is a 
mapping that occurs between two perspective images of a planar surface in the scene. 
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The computation of homographies plays an essential role in image registration and 
mosaicing." 

22. Therefore it would have been obvious to one having ordinary skill in the art at the 
time of the invention to use homography transformation matrices (as taught by 
Kanatani) to assist in the "deghosting" and warping performed in the mosaicing of 
Uyttendaele. Homography allows for the mapping between two points which would be 
necessary for the merging of images performed by Uyttendaele, this could result in a 
cleaner mosaicing with less ghosting. 

23. In regards to claim 14, Uyttendaele discloses in the paragraph starting at column 
7, line 41 that the "camera overlaps by at least 20 percent". This is about 25 percent. 

24. In regards to claim 15, as shown in claim 14 Uyttendaele discloses overlaps of at 
least 20 percent. As each image would overlap with two other images the total overlap 
for any one image would be about 40 percent. This is less than 85 percent of the field 
of view. 

25. In regards to claim 16, Kanatani discloses the use of homography matrices, 
which are based on landmark points of commonality as seen in the first full paragraph in 
the second column on page 426. 

26. In regards to claim 17, this claim has already been dealt with in claim 13. The 
entire idea of the invention disclosed by Uyttendaele is to fuse the plurality of frames of 
image pixel data into a single image, as shown in claim 13 this can be done using 
homography matrices. 
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27. In regards to claim 18, the subject of overlap percentages has been dealt with in 
claims 14 and 15. In regards to the outer perimeter edge, when Uyttendaele creates 
the panoramic image there is an upper and lower perimeter edge (as a panoramic 
image can be considered a ring, which has two edges) either could be the outer 
perimeter edge. 

28. In regards to claim 19, as shown in claim 14 Uyttendaele discloses overlaps of at 
least 20 percent. As each image would overlap with two other images the total overlap 
for any one image would be about 40 percent. 40 percent is about 35 percent. 

29. In regards to claim 20, Uyttendaele in the paragraph starting at column 7, line 41 
disclosed that a "camera rig preferably consists of multiple digital video cameras that 
are disposed in a back to back fashion such that their lenses each point in a radially 
outward direction and view a different portion of the surrounding scene." Uyttendaele 
does not disclose locating imaging devices at one or more additional installation sites. 
However as Uyttendaele is attempting to obtain a panoramic image it is not necessary 
for the method disclosed by Uyttendaele to have additional installation sites. 

30. In regards to claim 21, Uyttendaele discloses in the paragraph starting at column 
7, line 41 that "at least as many cameras as needed to provide a full 360 degree 
coverage of the scene including the desired overlaps would be employed." Thus 
inherently Uyttendaele discloses determining the limiting range of at least one of a 
plurality of imaging devices, as this information would be needed to determine the 
desired overlaps. 
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31. In regards to claim 24, Uyttendaele in the paragraph starting at column 7, line 41 
discloses that a "camera rig preferably consists of multiple digital video cameras that 
are disposed in a back to back fashion such that their lenses each point in a radially 
outward direction and view a different portion of the surrounding scene." Uyttendaele 
does not disclose locating imaging devices at one or more additional installation sites. 
However as Uyttendaele is attempting to obtain a panoramic image it is not necessary 
for the method disclosed by Uyttendaele to have additional installation sites. The search 
area is the panoramic image. In regards to the outer perimeter edge, when Uyttendaele 
creates the panoramic image there is an upper and lower perimeter edge (as a 
panoramic image can be considered a ring, which has two edges) either could be the 
outer perimeter edge. Finally, Uyttendaele discloses in the paragraph starting at column 
7, line 41 that the "camera overlaps by at least 20 percent". This is about 25 percent. 

32. In regards to claim 25, Uyttendaele discloses in the paragraph starting at column 
7, line 41 that "at least as many cameras as needed to provide a full 360 degree 
coverage of the scene including the desired overlaps would be employed." 

33. In regards to claim 26, as shown in claim 24 Uyttendaele discloses overlaps of at 
least 20 percent. As each image would overlap with two other images the total overlap 
for any one image would be about 40 percent. This is less than 85 percent of the field 
of view. 

34. In regards to claim 27, Uyttendaele discloses in the paragraph starting at column 
7, line 41 that "at least as many cameras as needed to provide a full 360 degree 
coverage of the scene including the desired overlaps would be employed." Thus 
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inherently Uyttendaele discloses determining the limiting range of at least one of a 
plurality of imaging devices, as this information would be needed to determine the 
desired overlaps. 

35. In regards to claim 28, Uyttendaele discloses in figure 3A a method for combining 
a plurality of frames of image pixel data. In step 300 Uyttendaele acquires multiple 
frames of image pixel data. In the paragraph starting at column 4, line 45 Uyttendaele 
discloses, "a camera 163 capable of capturing a sequence of images 164 can also be 
included as an input device to a personal computer." Any image inputted into a 
computer must be comprised of pixels. Uyttendaele then discloses in the paragraph 
starting at column 5, line 50 that each frame is "captured by a different cameras from a 
different viewpoint." Returning to figure 3A Uyttendaele discloses in step 306 that the 
image frames are combined (mosaic). This combined image is the search area 
(panoramic image). 

36. In regards to claims 31 - 35, claims 31 - 35 are the system claims of claims 24 - 
28 respectively. Please see these claims for the reasons of rejection. 

37. In regards to claim 38, Uyttendaele in the paragraph starting at column 7, line 41 
discloses that a "camera rig preferably consists of multiple digital video cameras that 
are disposed in a back to back fashion such that their lenses each point in a radially 
outward direction and view a different portion of the surrounding scene." The search 
area is the panoramic image. Finally, Uyttendaele discloses in the paragraph starting at 
column 7, line 41 that the "camera overlaps by at least 20 percent". This is about 25 
percent. 
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38. In regards to claim 39, as shown in claim 38 Uyttendaele discloses overlaps of at 
least 20 percent. As each image would overlap with two other images the total overlap 
for any one image would be about 40 percent. This is less than 85 percent of the field 
of view. 

39. In regards to claim 40, Uyttendaele discloses in the paragraph starting at column 
7, line 41 that "at least as many cameras as needed to provide a full 360 degree 
coverage of the scene including the desired overlaps would be employed." Thus 
inherently Uyttendaele discloses determining the limiting range of at least one of a 
plurality of imaging devices, as this information would be needed to determine the 
desired overlaps. 

40. In regards to claim 41 , Uyttendaele discloses in figure 3A a method for combining 
a plurality of frames of image pixel data. In step 300 Uyttendaele acquires multiple 
frames of image pixel data. In the paragraph starting at column 4, line 45 Uyttendaele 
discloses, "a camera 163 capable of capturing a sequence of images 164 can also be 
included as an input device to a personal computer." Any image inputted into a 
computer must be comprised of pixels. Uyttendaele then discloses in the paragraph 
starting at column 5, line 50 that each frame is "captured by a different cameras from a 
different viewpoint." Returning to figure 3A Uyttendaele discloses in step 306 that the 
image frames are combined (mosaic). This combined image is the search area 
(panoramic image). 
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41. Claims 11, 12, 22, 23, 29, 30, 36, 37, 42 and 43 are rejected under 35 
U.S.C. 103(a) as being unpatentable over Uyttendaele in view of Kanatani as applied to 
claim 1 above, and further in view of Hansen (6,081 ,606). 

42. In regards to claim 11, Uyttendaele in view of Kanatani discloses a method for 
obtaining a mosaiced video. Uyttendaele in view of Kanatani however, does not 
disclose the use of this method for surveillance. 

43. Hansen teaches in the paragraph starting in column 1 , line 46 that a video image 
can be used to track a moving object. 

44. Therefore it would have been obvious to one having ordinary skill in the art at the 
time of the invention to track a moving object (as taught by Hansen) in the video created 
by Uyttendaele in view of Kanatani. The mosaiced video created by Uyttendaele in view 
of Kanatani would be ideal for surveillance as a mosaiced image can cover much more 
of a search area in high resolution then a single camera can. Allowing for more accurate 
tracking and detection of abnormal situations. 

45. In regards to claim 12, Hansen teaches in the paragraph starting in column 1, 
line 46 that normal or abnormal motion is determined. "Abnormal motion is defined as 
motion within the scene, that is outside the channel." 

46. In regards to claim 22, Uyttendaele in view of Kanatani discloses a method for 
obtaining a mosaiced video. Uyttendaele in view of Kanatani however, does not 
disclose the use of this method for surveillance. 

47. Hansen teaches in the paragraph starting in column 1 , line 46 that a video image 
can be used to track a moving object. 
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48. Therefore it would have been obvious to one having ordinary skill in the art at the 
time of the invention to track a moving object (as taught by Hansen) in the video created 
by Uyttendaele in view of Kanatani. The mosaiced video created by Uyttendaele in view 
of Kanatani would be ideal for surveillance as a mosaiced image can cover much more 
of a search area in high resolution then a single camera can. Allowing for more accurate 
tracking and detection of abnormal situations. 

49. In regards to claim 23, Hansen teaches in the paragraph starting in column 1, 
line 46 that normal or abnormal motion is determined. "Abnormal motion is defined as 
motion within the scene, that is outside the channel." 

50. In regards to claim 29, Uyttendaele in view of Kanatani discloses a method for 
obtaining a mosaiced video. Uyttendaele in view of Kanatani however, does not 
disclose the use of this method for surveillance. 

51 . Hansen teaches in the paragraph starting in column 1 , line 46 that a video image 
can be used to track a moving object. 

52. Therefore it would have been obvious to one having ordinary skill in the art at the 
time of the invention to track a moving object (as taught by Hansen) in the video created 
by Uyttendaele in view of Kanatani. The mosaiced video created by Uyttendaele in view 
of Kanatani would be ideal for surveillance as a mosaiced image can cover much more 
of a search area in high resolution then a single camera can. Allowing for more accurate 
tracking and detection of abnormal situations. 
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53. In regards to claim 30, Hansen teaches in the paragraph starting in column 1, 
line 46 that normal or abnormal motion is determined. "Abnormal motion is defined as 
motion within the scene, that is outside the channel." 

54. In regards to claims 36 and 37, claims 36 and 37 are the system claims of claims 
29 and 30 respectively. Please see these claims for the reasons of rejection. 

55. In regards to claim 42, Uyttendaele in view of Kanatani discloses a method for 
obtaining a mosaiced video. Uyttendaele in view of Kanatani however, does not 
disclose the use of this method for surveillance. 

56. Hansen teaches in the paragraph starting in column 1, line 46 that a video image 
can be used to track a moving object. 

57. Therefore it would have been obvious to one having ordinary skill in the art at the 
time of the invention to track a moving object (as taught by Hansen) in the video created 
by Uyttendaele in view of Kanatani. The mosaiced video created by Uyttendaele in view 
of Kanatani would be ideal for surveillance as a mosaiced image can cover much more 
of a search area in high resolution then a single camera can. Allowing for more accurate 
tracking and detection of abnormal situations. 

58. In regards to claim 43, Uyttendaele in view of Kanatani discloses a method for 
obtaining a mosaiced video. Uyttendaele in view of Kanatani however, does not 
disclose the use of this method for surveillance. 
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Conclusion 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Christopher L Lavin whose telephone number is 703- 
306-4220. The examiner can normally be reached on M - F (8:30 - 5:00). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Leo Boudreau can be reached on (703) 305-4706. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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